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ABSTRACT 

The purpose of this paper is to demonstrate how the TRALE model (Technology-Rich Authentic Learning Environments] 
improved young urban learners' literacy skills. TRALE, designed as an early childhood education program, integrated 
authentic learning and the use of technology to increase disadvantaged children's literacy learning. The paper includes 
a description of how the model was implemented through the creation of a community of technology-enriched 
classrooms in an inner-city school context and provides student achievement data in language arts as evidence for 
TRALE's instructional effectiveness. 
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PREFACE 

"In a study of 20 'high income 1 countries, the US ranked 12th 
on literacy tests. Illiteracy has become such a serious 
problem in our country that 44 million adults are now 
unable to read a simple story to their child." (Education 
Portal, 2007). Statistics on illiteracy in the U.S. (based on data 
from the National Institute for Literacy, National Center for 
Adult Literacy, The Literacy Company, and the U.S. Census 
Bureau compiled by Education Portal) indicate that "50 
percent of adults cannot read a book written at an eighth 
grade level, 20 percent of Americans are functionally 
illiterate and read below a 5th grade level, and nearly half 
of all Americans read so poorly that they cannot find a 
single piece of information when reading a short 
publication." Additional data point out that "3 out of 4 
people on welfare can't read, 20 percent of Americans 
read below the level needed to earn a living wage, 50 
percent of the unemployed people who fall between the 
ages of 16 and 21 cannot read well enough to be 
considered functionally literate, and between 46 and 51 
percent of American adults have an income well below the 
individual threshold poverty level because of their inability 
to read." Social problems are also traced back to illiteracy 
as "3 out of 5 people in an American prison can't read, 85 
percent of juvenile offenders have problems reading, and 


approximately 50 percent of Americans read so poorly that 
they are unable to perform simple tasks such as balancing 
a checkbook and reading prescription drug labels." The 
shocking news is that elementary students' reading scores 
are used by prisons to project the number of beds 
necessary in the future. Illiteracy also has grave 
consequences financially for the U.S. "Illiteracy costs 
American taxpayers an estimated $20 billion each year. 
Illiteracy has been proven to cause children to drop out of 
school. Dropouts cost our nation $240 billion in social 
service expenditures and lost tax revenues." 

Introduction 

Historically, the poor have comprised a disproportionately 
large percentage of our nation's illiterate, and this problem 
shows no signs of abating. Today, poverty and illiteracy 
continue to be highly correlated, and the inability to 
communicate effectively (i.e., read, write, speak, and 
listen) continues to be one of the largest educational and 
social problems facing our nation. To compound the 
problem, the emergence of technology is rapidly creating 
new forms of "computer literacy." If lack of access to 
"traditional" forms of literacy has created a severe problem 
for the poor, lack of access to computer literacy, which is 
essential for communicating, functioning, and producing 
in today's world, will create an even greater gap between 
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mainstream and marginalized citizens. Data indicate that 
access to technology in our schools is not equally 
distributed across high-socioeconomic status (SES) and 
low-SES schools (e.g., Sutton, 1991). In Sutton's analysis of 
technology availability within our nation's schools, she notes 
that during the past three decades, the availability and use 
of technology in schools has increased dramatically. 
However, during that same period the technology gap 
between those schools educating children of poverty and 
those educating children from the middle and upper- 
middle classes has grown, and this trend has become 
even more pronounced in elementary schools. Further 
contributing to this growing chasm is the general belief that 
older rather than younger-age elementary children profit 
from technology use. This belief is evidenced repeatedly in 
urban elementary schools where the state-of-the-art 
technology (at least what there is of it) resides in upper 
grade classrooms and laboratories, while the older, 
donated, or otherwise throw-away technology funnels 
down for use into the earlier grades. In short, because of the 
compounding effects of low literacy, lack of access to 
technology, and the low priority of adequate technology in 
the early childhood grades, young poor children attending 
today's inner-city elementary schools are increasingly at risk 
in their ability to become competent, functioning citizens. 

One critical component of systemic reform in early 
childhood education is the design and implementation of 
effective, responsive instructional programs that meet the 
needs of young children in urban environments. The stated 
goal of TRALE (Technology-Rich Authentic Learning 
Environments) was to improve the literacy skills of young 
urban children through the creation of technology- 
enriched classroom environments. As an applied research 
project, TRALE had two objectives. Having assumed that 
children in urban schools have normal cognitive 
competencies, the principal investigators (Pis) designed 
this project to understand the reasons for these children's 
low levels of achievement on annual standardized tests 
used almost exclusively as the criterion for determining 
student success. Having gained some understanding of 
the variables that produced this overall low level of 
performance, the Pis altered the children's learning 
environments to make their learning more effective and 


promote higher achievement on these tests. In practice, 
TRALE worked toward these two objectives simultaneously. 
One goal was to uncover variables that contributed to the 
children's low performance levels, and this work 
proceeded as applied research that got built into the 
ongoing routine of schooling. The second goal was to use 
these and other research results to improve existing 
instructional practice. This work took the form of professional 
development and collaborative interactions with the 
teachers and administrative staff who participated in the 
project. 

This paper is comprised of two major sections: a description 
of TRALE followed by the student achievement data to 
substantiate the claim for TRALE's effectiveness in teaching 
language arts. The description of the program is divided 
into three parts: first, the paper provides an abbreviated 
theoretical rationale upon which TRALE rests, second, a 
concrete instantiation of the TRALE framework is described, 
and finally, the implementation of TRALE is summarized. 

The TRALE Project and Its Description 

This section describes the school, the children, and the 
political climate that affected the evolution and degree of 
implementation of TRALE. Subsequently, a brief history of 
TRALE's development is provided. Walker and Yekovich 
(1998, 1999) describe TRALE and its evolution in more 
detail. 

The School, The Children, and The Climate 

TRALE was implemented in early childhood education 
classrooms in one urban elementary school (called Target ) 
in a Mid-Atlantic city. Target Elementary is located in an 
area of high poverty, high crime, and rampant drug use. 
The school is located in an environment that is 
characterized by a lack of a thriving and varied 
community. The school's environment is basically literacy 
barren. 

The student population during the existence of the project 
was 100% African American. Over 97% of the children 
qualified for free breakfast and/or lunch. About 90% of the 
students lived in the public housing complexes, which 
immediately surrounded the school. The large majority of 
children came from single-parent families or resided with 
relatives. The instability of the home environment, coupled 
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with high poverty, created a great deal of variability in the 
importance placed on literacy within the home, These 
conditions also produced variability in the children's 
general readiness for school, especially with respect to 
academic, social, and motivational readiness. Not 
surprisingly, the children who attended Target tended to 
have behavioral difficulties and low levels of motivation to 
succeed academically. The high incidence of poverty 
produced substantial transience and was also related to 
the high truancy and dropout rates at the school. In short, 
the target population consisted of young, poor, urban 
minority children who were at high risk of educational 
failure. 

The context in which TRALE was implemented cannot be 
described as one that wholeheartedly promoted and 
supported new educational initiatives. Several obstacles 
hindered TRALE's evolution into a successful project. One of 
the many sources of frustration affecting TRALE's 
implementation originated from the school district itself. 
The actions of the executive officers stressed the system to 
a great extent and heightened the levels of fear and 
distrust among teachers and administrators within the 
schools. 

Target Elementary was identified as one of the 20 lowest 
performing schools within the school district. When 
standardized test scores confirmed Target as the lowest 
performing school in reading and mathematics, the 
school was mandated to adopt a whole school reform 
model called the Adaptive Learning Environments Model 
(ALEM) created at Temple University. The introduction of 
ALEM, to be co-implemented with TRALE, created 
additional tensions at the school and in fact made the 
implementation of TRALE more challenging. 

For years the early childhood grades were operating 
without a standard reading and math curriculum and little 
attention was paid to the alignment of curricular 
components with expected performance and formal 
assessment. In subsequent years teachers were expected 
to use the Houghton-Mifflin reading and math series that 
the state had adopted and purchased. The long-standing 
lack of alignment between the curriculum and assessment 
practices, coupled with the abrupt mandate to 


completely embrace and implement a poorly understood 
but new curriculum resulted in even more obstacles in the 
implementation of the TRALE project. 

A Brief History of TRALE 

The TRALE team's involvement with Target started when the 
Principal Investigators (PI) began volunteering at Target as 
tutors. From very early on, the Pis were struck by a paradox. 
Observations of the young children in informal, natural 
settings clearly showed them to be competent, motivated 
problem solvers and capable "information processors" 
(see Gagne, Yekovich, & Yekovich, 1993). Yet, equally clear 
was the observation that formal assessment of these 
children, especially when based on standardized tests, 
showed significantly lower performance. Why did young 
children who appeared to be cognitively competent in 
many informal settings fail to master formal cognitive skills 
and display the competence they had on formal, 
standardized tests? This question pushed the project 
forward and provided a context for creating an early 
childhood education program that more closely met the 
young children's needs in this urban school context. The 
next section describes TRALE's implementation. 

A Description of TRALE and Its Implementation 

What is TRALE? In essence TRALE is a principled framework 
for designing meaningful instruction for children. 
Meaningful instruction is created through the use of 
authentic, problem-based learning activities, and when 
appropriate, technology is utilized as a tool in the activities. 
So, for example, children may learn to alphabetize by 
creating an inventory list for their classroom store, and once 
the list is made, it may become a computerized 
spreadsheet that is used for real inventory and accounting 
purposes. 

At Target Elementary, TRALE was implemented as a 
community consisting of businesses and organizations. 
Each classroom took up a role (e.g., a store, newspaper, 
theater, etc.), and all the classrooms worked together as 
they used their own and each other's services. For instance, 
kindergarten actors (who couldn't read yet) performed in 
the theater, rented movies from the video store, went 
shopping to the store (and obtained help from 3 ,d grade 
shopping assistants), and listened to older students as they 
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were reading the school newspaper out loud for them. The 
community thrived (and students learned) through 
interacting within their classroom and across classes within 
the community. In other words, TRALE afforded the 
framework for activities that were based on the curriculum 
and standards of the school district and provided the 
cornerstone for children's meaningful learning (see Walker 
&Yekovich, 1999). 

The underpinnings of TRALE draw heavily from research in 
cognitive science, cognitive development, and 
sociocultural theory. Before describing a concrete 
instantiation of TRALE, a brief overview of the project's 
underlying theoretical assumptions follows in the next 
section that also includes the identification of five critical 
characteristics that a TRALE instructional setting possesses. 
Theoretical Underpinnings 

TRALE's approach to the teaching of literacy (i.e., reading 
and writing) has its roots in cognitive science (e.g., 
Anderson, 1987, 1993). Reading and writing, like all 
cognitive activities, are essentially problem-solving in 
nature and are best acquired in problem-based learning 
environments (see Anderson, 1995, p. 221). Like other 
complex cognitive skills, people develop expertise in 
reading and writing, and the course of that development 
parallels in many important ways the development of other 
types of cognitive competence (Scardamalia & Bereiter, 
1991). Similarly, just as experts in general rely on both 
automated and conscious skill components, expert 
readers and writers automate some parts of skills and retain 
other parts as conscious strategies (Just & Carpenter, 
1987). 

At the same time, TRALE is also based on the assumption 
that sociocognitive and sociocultural factors have a large 
impact on the organization and content of one's 
declarative and procedural knowledge (See Longer, 1987; 
Vygotsky, 1987). Since thought and action are functions of 
one's knowledge base, the social contexts that house the 
experiences of individuals directly shape the nature of a 
person's thoughts and actions, In this sense, one's 
cognitions are situated within particular social contexts. 
TRALE's Characteristics 

TRALE is characterized by five critical dimensions, such as 


Goal Directedness, Authenticity (i.e.. Contextualized 
Learning), and Community (or Shared Responsibility). 
Because of TRALE's special interests in young school-age 
children and in technology, the model includes two 
additional critical dimensions, i.e., Multiple Modes of 
Expression and Representation that is related to literacy 
development, and the Use of Technology. 
Goal-Directedness refers to the idea that human thoughts 
and actions have a purpose. One function a teacher or 
mentor serves is to help young children, often novices, see 
the goal - subgoal relations in real-world problems. It is 
critical for problem solvers (even young children) to know 
the goal of what they are doing and to understand how 
their current activity contributes to the achievement of that 
goal. If they have this understanding, their current task or 
activity makes sense to them - that is, the children see how 
the current "piece" fits into the big picture. 

Authenticity refers to the idea that the activities in a 
classroom are contextualized or situated in such a way that 
the students have some meaningful prior knowledge and 
experience they can use when they attempt to understand 
new concepts and solve new problems, For example, 
simulating a store in a classroom provides a familiar, 
meaningful, and authentic context for a child because of 
the child's prior knowledge and experiences with 
neighborhood stores. Thus, alphabetizing a list of inventory 
items becomes "real." The task is authentic. 

Shared Responsibility refers to the social nature of learning 
and provides a motivation for learning. The concept of a 
TRALE community attempts to capitalize on a primary 
human social need, namely that people want and need to 
be legitimate, contributing members of a group. The 
incentive to belong is utilized as a motivation for wanting to 
learn academic skills because one major way to be a 
legitimate, respected group member within TRALE 
classrooms and the community is to make academic 
contributions ("disguised" as TRALE activities) to the 
achievement of group goals. 

The fourth critical dimension is Multiple Modes of Expression 
and Representation. Because initially young children are 
constrained in the ways in which they can understand and 
communicate meaning-making events (e.g,, 
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kindergartners cannot read and/or write yet), their 
meaningful expression can be allowed to take other forms 
(e.g., drawing, acting, and singing) in addition to 
conventional forms like reading and writing, Similarly, 
instruction should provide multisensory opportunities for 
acquiring literacy skills so that children can develop 
enriched representations of their worlds, As children 
develop, they become more able to express their ideas in 
standard or formal ways, and so the progression through 
the TRALE program from kindergarten to third grade 
requires increasingly formal and standardized expressions 
of ideas (e.g., by the time a child is in third grade, she is 
expected to help edit and publish a newspaper where 
more standard, formal means of communication and 
expression are required). 

TRALE's fifth and final critical dimension is the Use of 
Technology because 

• multimedia technology is an efficient delivery system 
for multisensory input, so it encourages the 
development of multiple modes of expression and 
representation; 

• technology engages young children for long periods 
of time providing the practice they need to develop 
automated basic skills; 

• technology encourages children to work together to 
solve problems promoting shared responsibility; 

• technology provides structure and scaffolding (e.g., 
pull-down menus) that enable children to build goal- 
directed mental representations as they are working 
and solving problems; 

• well designed and pedagogically sound multimedia 
software programs allow young children to 
concentrate on higher level skills (e.g., constructing a 
meaningful message) while the computer takes over 
some parts of a resource consuming task (e.g., the 
computer puts the letters on the screen while the 
children concentrate on what they want to write), 

• technology can provide opportunities for children to 
practice small, prescribed sets of cognitive activities in 
motivating contexts, and 

• with the increasing importance of computer literacy in 


today's world, young children should have as much 
access as possible to the tools they will be expected to 
use in the future. 

TRALE's Implementation 

This section details how TRALE was conceptualized and 
implemented from its pilot implementation until Year 5. As 
mentioned before, TRALE has two delivery components, a 
technology component and an element dealing with 
authentic learning activities. In the following paragraphs, 
each component is described briefly. A more detailed 
description is provided by Walker & Yekovich (1998). 

TRALE's Technology Infrastructure 

Each participating classroom was equipped with several 
Apple computers with internet access and peripheral 
devices, such as a printer, a digital camera, a scanner, and 
headphones. Grade-level appropriate software packages 
allowed the children to develop their basic skills, such as 
reading, writing and math by providing spreadsheets, word 
processing and letter-sound matching activities, 
multimedia books and games, typing skills, interactive story 
writing and reading, word recognition activities, reading 
comprehension, phonics, and spelling. Other software was 
used for drawing, image editing, painting, coloring, 
scanning, block art, and games, such as checkers. 

The students used technology both in their skill 
development embedded in classroom activities as well as 
in community activities when they interacted (and learned 
together) with children from other classes. This aspect of the 
community learning is explored and described in more 
depth by (Cave, Yekovich, & Walker, 2010). 

Creating Authentic Literacy Environments: The 
Community 

The concepts of meaningfulness and authenticity entail 
the creation of classroom contexts in which the 
opportunities for acquiring literacy skills simulate situations 
based on the children's out-of-school life experiences. Thus, 
each classroom in the project simulated a specific 
organization that the children had experiences with, such 
as a general store or a post office. 

This article focuses on young students' performance in 
language arts. Just like most of the curriculum, the 
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language arts component of the curriculum was designed 
using problem-based situations that were intimately tied to 
each specific classroom role. For instance, the 3 rd grade 
journalists were responsible for all operations, functions, 
and activities related to publishing the school newspaper. 
In order to achieve their goals, they had to engage in 
standard-based literacy activities most of the time every 
day, For example, the young writers contacted potential 
interviewees with initial questions, brainstormed about and 
created interview protocols with appropriate questions, 
took notes while conducting the interviews, transcribed 
their notes on the computer, edited their papers in groups, 
and prepared the layouts. The language arts content 
standards were easy to cover as the authentic publishing 
activity naturally related to reading and writing skills. Some 
of the standards that the activities supported were the 
following: 

• Follow agreed-upon rules for class discussion and carry 
out assigned roles in self-run small group discussions, 
including posing relevant questions and building on 
the ideas of others, 

• Retell and paraphrase information shared orally by 
others, 

• Respond to questions with appropriate elaboration 
and detail (e.g., explain, amplify, expend), 

• Identify the purpose or main point and supporting 
details in text, 

• Identify the facts given in a text, and 

• Write stories that have a beginning, middle, and end 
and contain details of setting and characters. 

The publishing activities oftentimes pushed students to go 
beyond 3 rd grade expectations, as it was not sufficient in 
some situations to be able to only identify certain parts of a 
written text (e.g., purpose, main point, supporting details), 
but the students were supposed to write their own text and 
critique others' work also. 

All classrooms were set up in a fashion similar to the 
Newsroom: they all had a specific goal, problem-based 
activities that related to the classroom's function in the 
community, and technology to be used in accomplishing 
all literacy tasks. Learning occurred both inside the 


classroom and within the community. Oftentimes children 
practiced individual skills (e.g., conducting an interview) in 
their own classrooms before they attempted to utilize their 
newly acquired skills in another classroom. The interactive 
nature of learning was employed when classes were 
scheduled for community events, e.g., the journalists 
viewed a theatrical performance presented by the 
kindergarteners and wrote their critique in the school 
newspaper, or when the artists read the latest issue of the 
school newspaper. The essence of all these events was the 
improvement of skills in academic areas - later to be 
tested by standardized achievement tests. 

In sum, the teaching-learning process occurred in a 
meaningful way supported by technology. The problem- 
based activities were the foundation for learning where 

• interaction varied (both teacher-student and student- 
student), 

• scaffolding was used to expose problem structures 
and teach problem-solving strategies, 

• the learning environment evolved to meet the 
children's increasingly elaborate processing 
capabilities, and 

• the nature and purpose of each classroom role 
provided the overarching goal and motivation for 
the children to learn knowledge and skills. 

Though the purpose of the project was to create 
meaningful learning opportunities for the children, for 
which anecdotal evidence was abundant, the other 
essential goal was to increase students' achievement 
scores on the school district's standardized test. The second 
major part of this paper provides student standardized test 
data to demonstrate TRALE's effectiveness as an 
instructional paradigm in a low-performing school. 

The Instructional Effectiveness ofTRALE 

Throughout TRALE's implementation, student achievement 
data were systematically gathered from both formal 
(standardized test scores) and informal sources (writing 
samples, videotaped reading samples, alternative 
assessments of early reading skills, etc.): however, the 
ultimate test of the effectiveness of any instructional 
intervention today is whether it improves student 
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performance. Thus the main data source in this study is 
student achievement in language arts as measured by 
standardized tests. 

When TRALE was designed and first implemented in Target 
Elementary, its main goal was to increase student 
achievement significantly in its third year of 
implementation after its pilot. There are three reasons why 
TRALE was not expected to produce marked changes in 
student performance during the first two years of 
implementation following the pilot: (i) According to 
research, new programs on average are expected to 
increase students' test scores significantly only after two 
years of full implementation, (ii) TRALE's implementation was 
always delayed due to unfavorable conditions at the 
beginning of each school year, and (iii) the selected group 
of teachers' participation in TRALE was first voluntary and 
became obligatory beginning only in the 4 th and 5 th project 
years. 

No matter how ingenious an educational invention, 
program, or project may be, it can produce significant 
changes in academic progress only when it is fully and 
consistently implemented to a high degree. In order to 
make a dramatic difference in students' achievement, 
TRALE had to be highly implemented. Thus, the teachers 
were categorized according to their degrees of 
implementation. During the first three years of the project 
including the pilot year, teachers' degree of program 
implementation was assessed according to a complex 
system including (1) the project coordinator's observations 
and her anecdotal evaluation of classroom roles, (2) 
teachers' meeting attendance, (3) number of teacher 
journal entries submitted, (4) quality of teacher journal 
entries, and (5) the number of planned community 
activities and events accomplished. At the end of the 3 rd 
year, the project team devised a 118-item instrument 
called the Degrees of Implementation for the TRALE project 
(TRALE DOI) in order to formally assess program 
implementation. The TRALE DOI data using this new 
instrument became available in the 4 th year. From that 
point on, it was more feasible to account for student growth 
knowing each teacher's implementation profile. The next 
section includes data analyses from each project year, 


Student Achievement in the Pilot Year 

In the pilot year Grade Equivalent Scores (GES) were used to 
report student achievement data. GES measures gauge 
"growth within student" by indicating a student's grade level 
year and month of achievement. For instance, a score of 
2.1 means that a student is working at a second grade level 
in the first month of school. If a student's score changes 
from 2.1 to 2.6 in a 5-month period, one can infer that the 
student is progressing at a normal rate. "Within student" 
measures remove random differences of no particular 
interest (e.g., differences in ability between classes) and 
provide a more reliable gauge of change within an 
individual overtime. 

As published in an earlier paper (Cave, Yekovich, & Walker, 
2010) during the pilot implementation of the program the 
2 nd graders made significant gains in language arts from 
Year 1 to Year 2 as measured by GES on the state's 
standardized achievement test. Gains in the 2 nd grade 
project class were tenfold in vocabulary (project class = 
.74, non-project = .07) and language expression (project 
class = .7, non-project = .07) whereas gains were threefold 
in reading comprehension (project class = 1.01, non¬ 
project = .36) using Grade Equivalent Scores. 

Gains in the 3 ,d grade classes were challenging to measure 
due to reliability problems with pretest data, however, it was 
confirmed that the 3 rd grade project class made a one- 
year gain in language mechanics (project class = 3.5, 
non-project = 2.5) using GES and showed overall improved 
performance in reading, spelling, and language total. 
Student Achievement across Second and Third Years 

All the teachers were divided up into three groups: (1) high 
implementing TRALE teachers, (2) low implementing TRALE 
teachers, and (3) non-project teachers. It was assumed 
that students' achievement would be significantly higher in 
the high implementing teachers' classes compared to low 
implementing and non-project teachers' classes. In these 
two school years there were four high implementing, four 
low implementing, and seven non-project teachers. In 
each year TRALE was implemented, the size of the school 
allowed for at least one TRALE and one non-TRALE 
classroom on each grade level. However, complete and 
reliable data were not available all the classes. 
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Kindergarten teachers were excluded from this analysis for 
kindergartners were not tested by the state's standardized 
test. In the 2 nd and 3 rd years of the project the subtests 
reported by the state were different from the ones reported 
in the first year. Thus, the total reading scores were 
analyzed. Table 1 displays student achievement scores in 
(i) high implementing, (ii) low implementing, and (iii) non¬ 
project teachers' classes. 

As the state changed the reporting format of student 
achievement data from GES to Normal Curve Equivalent 
(NCE) scores, the data analyses in this study also included 
NCE scores from this year on. The NCE format with its 
percentile-like rankings allowed for meaningful analyses of 
growth from fall to spring and comparisons to the national 
norms. Table 1 indicates the students' total reading scores 
in an NCE format as well as the average in each group of 
high implementing, low implementing, and non-project 
teachers. 

The data analyses revealed several noteworthy results. First, 
the degree of program implementation was clearly related 
to growth in the students' total reading scores. Average 
growth in students' total reading scores in the high 
implementing teachers' classes was tenfold compared to 
low implementing teachers' classes and sevenfold 


compared to non-project teachers' classes. 

Second, students in the high implementing teachers' 
classes seemed to begin the school year at a lower 
reading level (21 st %tile) when compared to children in the 
low implementing (29 m %tile) and non-project (28 th %tile) 
classes. However, by the end of the year, children with lower 
NCE reading scores at the beginning of the year slightly 
outperformed those children who started the year in a 
better position. 

In sum, even though TRALE was not expected to increase 
achievement scores on the standardized tests during the 
first three years of its existence, it was a successful project in 
doing so even in the first year of its pilot implementation 
and every school year thereafter. Students' standardized 
test scores indicated that TRALE - when highly implemented 
- significantly increased student achievement. 

Student Achievement in the Fourth Year 

Similarly to analyses in the previous school years, the 
student achievement data were analyzed by dividing up 
all the teachers in 1 st , 2 nd , and 3 ,d grades according to their 
degrees of implementation of TRALE. Again, the 
kindergarten teachers were not included in the analysis 
because the standardized test was not administered to 


Group of Classes According 
to Degrees of Implementation 

Project Year 

Total Reading Fall 
Semester and 
Average in Group 

Total Reading Spring 
Semester and 

Average in Group 

Growth in 
NCE Points 

Average Amount of Growth in 
NCE Points across Classes in 
Each Group 

High Implementing Teachers 






Teacher 11 

Year 2 

22.8 

24.6 

1.8 


Teacher 11 

Year 3 

21.2 

37.5 

16.3 

10.9 

Teacher 2 

Year 2 

19.9 

31.0 

11.1 


Teacher 12 

Year 3 

21.2 

35.6 

14.4 




21.3 

32.2 



Low Implementing Teachers 






Teacher 2 

Year 3 

26.8 

30.0 

3.2 


Teacher 3 

Year 2 

37.2 

24.2 

-12.9 


Teacher 3 

Year 3 

30.4 

37.6 

7.2 

1.1 

Teacher 9 

Year 3 

20.8 

27.8 

7.0 




28.8 

29.9 



Non-Project Teachers 






2 nd grade (A) 

Year 2 

21.1 

20.4 

-.7 


2 nd grade (A) 

Year 3 

21.2 

33.6 

12.4 


2 nd grade (B) 

Year 2 

22.8 

22.1 

-.7 


2 nd grade (B) 

Year 3 

29.1 

37.4 

8.3 

2.7 

l nd grade (A) 

Year 2 

32.5 

25.8 

-7.2 


l nd grade (A) 

Year 3 

31.0 

42.6 

11.6 


l nd grade (B) 

Year 2 

39.5 

34.5 

-5.0 




28.2 

30.9 




Table t. Growth in Students' Total Reading Scores on the State's Standardized Achievement Test in 6 Months in Each Teacher's 
Class and Average of Growth in the Total Reading across Classes in the 3 Groups of Varying Degrees of Implementation 
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Group of Classes According Project Year Grade Year Total Reading fall 
to Degrees of TRALE Semester and 

Implementation Average in Group 


Total Reading Spring Growth in NCE Average Amount of 
Semester and Points Growth in NCE Points 

Average in Group across Classes in 

Each Group 


Medium Implementing 
Teachers 


Teacher 2 

Year 4 

3 

40.11 

38.80 

-1.31 


Teacher 8 

Year 4 

2 

28.48 

37.08 

8.6 

3.64 




34.29 

37.94 

7.29 


Low Implementing 
Teachers 







Teachers 

Teacher 3 

Year 4 

1 


43.69 



Teacher 6 

Year 4 

1 

36.12 

52.16 

16.04 


Teacher 9 

Year 4 

1 

25.92 

44.63 

18.71 

3.44 

Teacher 4 

Year 4 

2 

31.00 

51.75 

21.75 


Teacher 7 

Year 4 

3 

34.55 

33.17 

-1.38 





31.89 

45.08 

13.78 



Table 2. Comparison among Medium and Low Implementing Teachers in Terms of Average Amount of Change from the Beginning to the 
End of the Fourth Year on Each Grade Level on the Standardized Achievement Test Using Normal Curve Equivalents (NCEs) 


kindergartners. Unfortunately, Target did not have any high 
implementing TRALE teachers in the fourth school year. 
Table 2 displays student achievement scores in (i) medium 
and (ii) low implementing teachers' classes. 

Because the total reading scores were reported in NCE 
format, just like in previous years' analyses, they were used 
again to calculate growth in this academic year. Table 2 
indicates the average amount of growth in students' total 
reading scores in an NCE format as well as the average in 
each group of medium and low implementing teachers. 

The data analyses disclosed some unexpected results in an 
interesting pattern, First, the data indicated that degree of 
implementation was not clearly related to students' total 
reading scores on the standardized test. In one of the 
medium implementers' classroom the students' growth was 
substantial (from the 28 th %tile to the 37 th %tile); however, 
the growth in the other medium implementing teacher's 
classroom was not observable. This was the most surprising 
part of the analysis since this "veteran TRALE teacher" 
always had very observable growth patterns in her 
classroom as measured on standardized tests. Second, 
even though students in the low implementing teachers' 
classes seemed to begin the school year at a lower 
reading level (31 st %tile on average) when compared to 
children in the medium implementing teachers' classes 
(34 th %tile), by the end of the year these children in most low 
implementing teachers' classes outperformed children in 


the medium implementing teachers' classes as measured 
by the standardized test. In other words, growth in most of 
the low implementing teachers' classes was more 
prevalent than in the medium implementing teachers' 
classes using standardized achievement test scores. 
However, the average amount of growth across classes in 
the 2 groups showed the same amount of gain (3.64 in 
NCE points for the medium implementing teachers and 
3.44 in NCE points for the low implementing teachers). 

Table 3 demonstrates how the percentage of the students 
were categorized according to performance standard 
levels, such as below basic=bb, basic=b, proficient=p, 
and advanced=a. 

Since performance standard levels were not assigned to 1 st 
graders during the fall administration of the standardized 
test, change in terms of performance standard levels 
could not be calculated for 1 st grade students. The data on 
performance standard levels in the 2 nd and 3 [d grade 
classes indicated that compared to low implementing 
teachers, medium implementing teachers were less 
successful in moving children out of the below basic 
category by the end of the year. Also the medium 
implementing teachers had more children moving to 
lower performance standard levels as the end of the year 
approached. 

As stated at the beginning of this paper, TRALE was 
expected to produce significant changes in students' 


i-manager's Journal on School Educational Technology, Vol. 5 • No. 4 • March - May 2010 


9 










RESEARCH PAPERS 


Group of Classes Project Year Grade Level Performance Standards in % Performance Standards in % of Change in % of Students in Various 
According to Degrees of Students in Each Category Students in Each Category Categories using Performance 

of TRALE Fall Semester Spring Semester Indicators 

Implementation 


Medium Implementing 
Teachers 



bb 

b 

P 

a 

bb 

b 

P 

a 

bb 

b 

P 

a 

Teacher 2 

Year 4 

3 

25.0 

56.3 

18.7 

0.0 

52.9 

29.4 

17.7 

0.0 

27.9 

-26.9 

-1.0 

0.0 

Teacher 8 

Year 4 

2 

33.3 

55.6 

11.1 

0.0 

50.0 

38.8 

5.6 

5.6 

16.7 

-16.8 

-5.5 

0.0 

Low Implementing 
Teachers 



bb 

b 

P 

a 

bb 

b 

P 

a 

bb 

b 

P 

a 

Teachers 

Teacher 3 

Year 4 

1 

- 

- 

- 

_ 

20.0 

50.0 

30.0 

0.0 

- 

- 

- 

. 

Teacher 6 

Year 4 

1 

- 

- 


- 

0.0 

69.2 

30.8 

0.0 

- 

- 

- 

- 

Teacher 9 

Year 4 

1 

- 


“ 

- 

14.3 

64.3 

21.4 

0.0 

- 



- 

Teacher 4 

Year 4 

2 

9.1 

77.3 

13.6 

0.0 

9.1 

68.2 

22.7 

0.0 

0.0 

-9.1 

9.1 

0.0 

Teacher 7 

Year 4 

3 

50.0 

27.8 

22.2 

0.0 

55.6 

27.8 

16.6 

0.0 

5.6 

0.0 

-5.6 

0.0 


‘Note: Instruction is considered successful if (1) the Change number is negative in the below basic category from the fall to the spring semester, which means 
that the % of students decreased in the below basic category from the fall to the spring semester and (2) simultaneously if the Change number is positive in the 
below and proficient categories from the fall to the spring semester, which means that the % of students increased in those categories. 


Table 3. Comparison among Medium and Low Implementing Teachers in Terms of Average Amount of Change from the Beginning to 
the End of the Fourth Year on Each Grade level on the State’s Standardized Achievement Test based on the % of Students in 

Various Categories Using Performance Indicators 


performance in the fourth school year. Because in the past 
there were reasons for concern about the reliability of the 
data gathered by the standardized tests at Target, an 
alternative method of assessment was conducted in a 3 rd 
grade classroom in order to determine whether the 
technology-rich authentic learning environment indeed 
improved young urban children's literacy more than the 
traditional literacy instruction (Cave, Yekovich, & Walker, 
2010). More specifically, the goal of that study was to 
determine whether there were any qualitative and/or 
quantitative differences in essays written by the 3rd grade 
students in the TRALE Newsroom and a control class. The 
prose analysis method used in the study allowed for 
examining the deep structure of each text and provided a 
detailed analysis of students' writing on macro and micro 
levels simultaneously Data on the children's organizational, 
paragraph-level thinking in their writing indicated that the 
TRALE students produced longer essays, organized and 
connected their essays more successfully on local and 
global levels, and composed more complex and goal- 
directed essays. This tendency continued throughout the 
school year, and the difference between the two classes' 
performance became wider on all variables. 

This extensive study provided sound evidence for TRALE's 
effectiveness on children's literacy development that was 
not captured by standardized tests either due to the nature 


of standardized tests or the unreliability of test scores in 
particular classes. For a more complete analysis of TRALE's 
effect on students' writing, see (Cave, Yekovich, & Walker, 
2010 ). 

Student Achievement in the Fifth Year 

In order to be consistent with previous years' analyses, 
growth analyses continued to utilize NCE formats to explore 
students' reading achievement in the fifth year, Similarly to 
the fourth year, degrees of implementation data were used 
to categorize teachers into three groups of program 
implementation. Again, the kindergarten teachers were 
not included in the analysis because the state's 
standardized test was not administered to kindergartners. In 
the fifth year there were 1 high, 2 medium, and 3 low 
implementing teachers at Target. There was also one 
teacher who began the school year after September and 
thus was not involved in implementing TRALE (non-project 
teacher). Table 4 displays student achievement scores in (i) 
high implementing, (ii) medium implementing, (iii) low 
implementing, and (iv) non-project teachers' classes. 
More specifically, Table 4 indicates students' total reading 
scores in an NCE format as well as the average in each 
group of high, medium, low implementing and non¬ 
project teachers' classes. 

Data analyses revealed some very interesting results. First, 
the data indicated that degree of implementation was not 
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Group of Classes According 
to Degrees of TRALE 
Implementation 

Project Year 

Grade Level Total Reading Fall 
Semester and 
Average in Group 

Total Reading Spring 
Semester and 

Average in Group 

Growth points 
in NCE 

Average Amount of Growth 
in NCE Points across 

Classes in Each Group 

High Implementing Teachers 







Teacher 10 

Year 5 

2 

28.24 

30.91 

2.67 

2.67 




28.24 

30.91 

2.67 


Medium Implementing Teachers 







Teacher 9 

Year 5 

1 

28.08 

60.63 

32.55 


Teacher 7 

Year 5 

3 

35.27 

55.44 

20.17 

13.18 




31.67 

58.03 

26.36 


Low Implementing Teachers 







Teacher 6 

Year 5 

1 

45.64 

50.87 

5.23 

2.98 

Teacher 11 

Year 5 

2 

25,56 

34.13 

8.57 


Teacher 4 

Year 5 

3 

34.87 

47.92 

13.05 





35.35 

44.30 

8.95 


Non-Project Teachers 







Teacher 12 

Year 5 

2 

23.52 

41.51 

17.99 

17.99 




23.52 

41.51 

17.99 



Table 4. Comparison among High, Medium and Low Implementing Teachers as well as Non-Project Teachers in Terms of Average Amount 
of Change from the Beginning to the End of the Fifth Year on Each Grade Level on the State’s Standardized Achievement Test Using 

Normal Curve Equivalents (NCEs) 


consistently related to students' total reading scores on the 
standardized test. Growth in student achievement in the 
highest implementing teacher's class (2,67 NCE points) 
occurred to a much lower degree than in medium 
implementing or non-project teachers' classes (13.18 NCE 
points and 17,99 NCE points respectively). According to the 
standardized test scores, medium implementing teachers 
were more successful in supporting student progress in 
reading than the high implementing or low implementing 
teachers whose students showed similar growth patterns 
(2.67 NCE points and 2.98 NCE points respectively). The fifth 
year student achievement data showed that the non¬ 
project teacher who was hired later in fall of the fifth year 
and was also a novice teacher produced the most growth 
in student reading. In this school year concerns about the 
reliability of student achievement data continued to exist. 

Table 5 demonstrates how the percentage of the students 
was categorized according to performance standard 
levels, such as below basic (bb), basic (b), proficient (p), 
and advanced (a). Since performance standard levels 
were not assigned to 1 st graders during the fall 
administration of the standardized test, change in terms of 
performance standard levels was not calculated for 1 st 
grade students. The data on performance standard levels 
in the 2 nd and 3 ,d grade classes indicated that the novice, 
non-project teacher was the most successful in moving 


children up from the below basic category. Almost 20% of 
her children moved out from the below basic category, 
more than 8% of her children moved to below, and more 
than 11 % of her children became proficient from October 
when she was hired to April when the standardized test was 
administered to the children. Again, the reliability of the 
data is dubious. The data on performance standard levels 
also showed that second to the non-project teacher, the 3 rd 
grade medium implementing teacher (since the other 
medium implementing teacher had no performance level 
data on her children in first grade) was more successful 
than the high and low implementing teachers in moving 
children upward to higher proficiency levels. The medium 
implementing teacher moved more than 11 % of her 
children out of the below basic category, and almost all of 
her other children moved to proficient (47.6%) or 
advanced (19.0%) levels. Student achievement data in 
one of the two low implementing teachers' classes 
exhibited an interesting pattern. The 3 rd grade low 
implementing teacher, similarly to the medium 
implementing teacher, moved almost half of her children 
to the proficient reading level, which resulted in a more 
than 60% reduction of number of students in the below 
category. 

In sum, the fifth year brought mixed results. TRALE was 
expected to result in a substantial increase in student 
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Group of Classes Project Grade Performance Standards in % of Performance Standards in % of Change in % of Students in Various 

According to Degrees Year Level Students in Each Category Students in Each Category Categories using Performance 

of TRALE Implementation Fall Semester Spring Semester Indicators 


High Implementing 
Teachers 

Year 5 

2 

bb 

b 

P 

a 

bb 

b 

P 

a 

bb 

b 

P 

a 

Teacher 10 



36.4 

36.4 

27.2 

0.0 

69.0 

25.0 

6.0 

0.0 

32.6 

-11.4 

-21.2 

0.0 

Medium Implementing 
Teachers 

Year 5 


bb 

b 

P 

a 

bb 

b 

P 

a 

bb 

b 

P 

a 

Teacher 9 


1 

- 

- 

- 

- 

42.0 

42.0 

16.0 

0.0 

- 

- 

- 

- 

Teacher 7 


3 

35.0 

45.0 

0.0 

0.0 

23.8 

9.5 

47.6 

19.0 

-11.2 

-35.5 

47.6 

19.0 

Low Implementing 
Teachers 

Year 5 

1 

bb 

b 

P 

a 

bb 

b 

P 

a 

bb 

b 

P 

a 

Teacher 6 



- 

- 

- 

- 

5.9 

64.7 

29.4 

0.0 

- 

- 

- 

- 

Teacher 11 


2 

38.5 

53.8 

7.7 

0.0 

46.1 

53.9 

0.0 

0.0 

7.6 

.1 

-7.7 

0.0 

Teacher 4 


3 

22.2 

72.2 

5.6 

0.0 

37.0 

10.5 

52.6 

0.0 

14.8 

-61.7 

47.0 

0.0 

Non-Project Teachers 

Year 5 

2 

bb 

b 

P 

a 

bb 

b 

P 

a 

bb 

b 

P 

a 

Teacher 12 



47.0 

53.0 

0.0 

0.0 

27.8 

61.1 

11.1 

0.0 

-19.2 

8.1 

11.1 

0.0 


‘Note: Instruction is considered successful if (1) the Change number is negative in the below basic category from the fall to the spring semester, which means 
that the % of students decreased in the below basic category from the fall to the spring semester and (2) simultaneously if the Change number is positive in the 
below and proficient categories from the fall to the spring semester, which means that the % of students increased in those categories. 

Table 5. Comparison among High, Medium and Low Implementing Teachers as well as Non-Project Teachers in Terms of Average 
Amount of Change from the Beginning to the End of the Fifth Year on Each Grade Level on the State Standardized Achievement 
Test Based on the % of Students in Various Categories using Performance Indicators 


achievement as measured by the state's standardized test; 
however, the data indicated that (I) growth in a novice, 
non-project teacher's classroom was the most substantial 
and (ii) when compared to high or low implementing 
teachers, medium implementing teachers on average 
were more successful in raising student achievement. The 
results of this analysis were not completely surprising, Every 
data analysis is conducted on data that are provided to 
researchers. If data are contaminated, the analysis will be 
affected. 

Lessons Learned 

TRALE's history at Target was nothing but adventurous. 
TRALE's ability to impact K-3 learning was demonstrated in 
highly implementing teachers' classes during the initial 
implementation of the program. Project faculty learned 
invaluable lessons regarding the implementation of a 
program in a context that was not always conducive to 
teaching, learning, and innovations. It is hoped that some 
of the lessons and insights gained by this project's staff assist 
others who wish to put into operation programs to increase 
student learning in urban school districts. 

Some of these lessons include the realization that no matter 
how innovative a research-based early childhood 
program may be, top-down programs, mandated by 


school districts do take priority and impact the time, 
resources, and efforts teachers are able to devote 
(regardless of their effectiveness). Little grass-roots efforts in 
the form of a partnership between a university and a local 
school increase volunteer teachers' ownership and 
support, but external forces play a larger role in determining 
which program gets implemented. Many barriers from 
external (and internal) sources may appear - including 
leadership at the school, district, and federal levels. If 
administration does not formally support the introduction 
and maintenance of a program, chances for success are 
little in the long run. The best course of action would include 
a plan for communicating and collaborating with not only 
the school principal, but the teachers' union, bureaucratic 
school districts, and coordinators of other programs that 
may be mandated by the school district. 

Another lesson is that teachers who volunteer for a program 
have a better ownership and seek out opportunities to 
increase program implementation. They need extensive 
support, which can be provided by one or more full-time 
program coordinators who are at the teachers' disposal at 
all times. After the pilot, program coordinators can 
decrease the amount of help depending on how 
independent teachers are and how well they understand 
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the principles of the new program. Degrees of program 
implementation, standardized student achievement as 
well as anecdotal data may inform program coordinators 
about how much to scaffold each teacher in her 
implementation. Even if program support is reduced, 
many teachers may need emotional support, which 
means visits to the classroom and opportunities to discuss 
learning events and their effectiveness. 

Getting support from the school leadership as well as the 
community is certainly helpful and increases the chances 
of parental involvement. Parents who are invited to 
participate in regular community events that are 
meaningful for both their children and themselves feel 
more connected and can support learning at home more 
efficiently. 

Conclusion 

Teaching is hard in urban America. Besides the lack of 
resources, teachers are supposed (mandated) to 
implement more than one program forced on them 
oftentimes haphazardly. No matter how good or how well 
supported by a core of teachers a grass-root effort is, it 
cannot survive in a climate where different mandates 
coming from the top administration are in conflict. It is hard 
for the teachers to maintain their allegiance to a program 
(even if they know it is powerful) if they are being told 
constantly to comply with all requirements (but those 
requirements do not apply to the grass roots effort). 
Considering all the obstacles that TRALE had to endure, it is 
astounding that TRALE not only survived in an urban school, 
but it positively impacted young urban learners 1 learning. 
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